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PREFACE 



In July of 1967, The Center sponsored a research planning 
seminar in distributive education. Participating in the sem- 
inar were national leaders in distributive education and re- 
source personnel from educational research and the behavioral 
sciences. While the primary purpose of the seminar was to 
develop a systematic procedure for evolving long-range research 
plans in distributive education, it is believed that the mode 
of attack and strategy has relevance for other areas. 

During the two and one-half day meeting, several papers 
were presented to the group. This report is a compilation of 
the papers utilized during the seminar. We believe they will 
be of use not only to distributive educators, but to personnel 
in other areas of vocational education as well. 

Recognition is due Neal E. Vivian and Edward T. Ferguson 
of The Center staff for their contribution to the development 
of the report. 



Robert E, Taylor 
Director 

The Center for Vocational 
and Technical Education 




INTRODUCTION 



A review of the research in distributive education com- 
piled in 19661 revealed research in this area to be almost 
entirely descriptive in nature. The reviewers were able to 
locate only one experimental study and one comparative study 
in distributive education. A narrow scope of research also 
proved evident, with 75 percent of the studies reported being 
concerned with four major areas: examination of teaching 

methods, guidance and personnel services, curriculum, and 
evaluation. Relatively little treatment had been given to 
areas such as teacher education, philosophy and objectives, 
facilities and equipment, administration, supervision, and 
instructional materials. 

It would be fair to say that slow progress has been made 
regarding research in distributive education and that the vast 
majority of research completed over the past 30-odd years has 
been narrow in focus, confined to minor problems, and, for the 
most part, local or regional in scope. Further, almost no ef- 
fort has been made to coordinate research in the field or to 
direct research resources to problems of major significance. 

Although the quality of descriptive research has shown 
great improvement over the last decade, it is obvious that 
distributive education cannot expand and improve on the 
strength of this form of research alone. A major effort 
must be made to conduct a variety of types of research that 
will add to knowledge in the field as well as evaluate ex- 
isting beliefs. 

The effort to enhance research achievement in distribu- 
tive education would be facilitated if major problem areas 
were identified and research were established. The resulting 
structure could be useful to individual investigators and 
make possible the coordination of research resources from in- 
stitutions and agencies across the nation in a focused attack 
on specific problem areas. Coordination of agency and in- 
stitutional research resources might be accomplished through 
existing state and university structures by an appropriate 
national agency. 



*Meyer, Warren G., and Logan, William B., Review and Synthesis 
of Research in Distributive Education , The Center for Voca- 
tional and Technical Education, The Ohio State University, 

Co I umbus , 1966. 
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RESEARCH PLANNING SEMINAR 



SEMINAR OBJECTIVES 



A research planning seminar was held at The Center dur- 
ing July 1967 with the purpose of formulating long-range plans 
for research development in distributive education. Three 
objectives of the seminar were: 

0 To discuss the present status of research in distribu- 
tive education. 

0 To establish research priorities in distributive edu- 
cation. 

0 To develop a framework for long-range planning of re- 
search activities through a research matrix. 



SEMINAR OVERVIEW 



The seminar was conducted on the premise that critical 
problem areas and research priorities could be identified by 
a group of leaders in distributive education and educational 
research. These leaders, not only familiar with the problems 
of distributive education, but also competent in research, 
would propose research priorities and develop a matrix which 
could facilitate long-range research planning based upon es- 
timated priorities and availability of funds. To this end, 
members of The Center's Distributive Education Advisory 
Committee joined with other educational research consultants 
and with The Center staff in a two and one-half day meeting. 
(See Appendix 3 for seminar participants.) Several back- 
ground papers, presentations, and other materials were made 
available to the group prior to the seminar. The following 
papers were selected for inclusion in this publication to 
provide a rationale for, as well as a better understanding 
of, the suggested distributive education research matrix 
presented in this report. 




PAPERS RELATING TO 
THE SEMINAR 




Some Problems of Design 
for Educational Research 



WILLIAM W, FAEQUHAE 
Michigan State University 



It seems to me that the major problems of educational research 
design are related to 1) theory development, 2) oversimplified 
designs, 3) manipulation and control of variables, 4) adequacy of 
criteria, 5) ethics, and 6) funding. 



THEORY DEVELOPMENT 

Some educational researchers have few problems with theory 
because they dismiss it as unnecessary. They contend that au 
facts accumulate, theory will evolve to explain them. Furthemore , 
they maintain that the rush to develop theory and the seeking of 
verification only prejudices the researcher's inquiry. Others 
maintain that theory in the social sciences is so loosely con- 
structed that it has limited value in explaining empirical phe- 
nomena. 

The case for the use of theory as a research base rests on 
its 1) explanatory, 2) interpolation-extrapolation, and 3) 
heuristic value. 

If the educational researcher decides that he will use theory 
to build models and generate hypotheses, he faces the problem of 
choosing which theory. In educational research, ultimately, the 
individual is the unit of study. The researcher may choose to 
emphasize only one aspect of the individual, but he cannot forget 
that the variables being studied are constantly being modified by 
other aspects which the theory may ignore. In short, he has a 
complex set of variables to consider. For example, when the 
researcher employs learning theory largely borrowed from psychology 
he usually ignores the cultural aspects of the child's background, 
which might explain his results equally well. Every theory 
magnifies certain variables, oversimplifies others, and completely 
ignores others. The net effect is that the educational researcher 
is always left fretting over his lack of closure in interpreting 
research results. 
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The researcher who attempts to use theory as a guideline for 
hypothesis generation and testing encounters a number of problems. 
For most social science theories, constructs are not sufficiently 
defined so that many intervening variables can also account for 
the research findings. Often we oversimplify by defining single 
traits or characteristics and ignoring the relationships which 
exist among other constructs and those we feel are important. 

That is, one might predict "x" from "y" providing "all other things 
being equal" and "j" or "u" did not also occur. We have been 
extremely limited in providing information about the "j*s" and the 
"u*s." 



OVERSIMPLIFIED DESIGNS 

The most common error evidenced in educational research is 
assessing complex acts (such as teaching or counseling) with 
rudimentary designs. It is a safe generalization to say that 
certain techniques work for certain individuals with certain per- 
sonalities. The generalization holds for both the experimenter 
and the subject of the experiment. Yet rarely does the research 
design take this aspect of human beings into account. N subjects 
are subjected to treatment X from experimenter Y with disappoint- 
ing, non-significant results. We would not think of doing such 
a thing in our daily living, but somehov; this is all forgotten 
when we design a research study. A relevant, but not directly 
related, article to the field of education by Kiesler (1966) 
neatly delineates the above problem. 



MANIPULATION AND CONTROL OF VARIABLES 

Once the researcher passes, for the moment, the theory stage, 
he is faced with the practical problems of 1) selecting adequate 
samples and 2) defining a plan of action which will randomize 
chance errors, minimize or assess the effects of systematic errors, 
and maximize the chances that "x" is actually functionally related 
to "y." An excellent guide to many educational design problems 
can be found in the recent publications of Campbell and Stanley, 
Experimental and Quasi-Exp erimental Designs for Research , (1966). 



SAMPLE SELECTION 

At some point the educational researcher has to decide how 
big his sample will be. The problem becomes one of juggling cost, 
available sample, representativeness, and the researcher's 
conservative-grandiose tendencies. As you are well aware, the 
most common fallacy is the assumption that big samples are good 
samples. (See Appendix 1 for Sampling Chart.) 
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MINIMUM LIMITS 



There are some minimum limits that educational researchers 
try to keep in mind. In classical experimental studies, it is 
nice, but not absolutely mandatory, if the smallest sub-class for 
any one sample hovers around 30 individuals. This observation 
is a "seat of the pants" rule developed by reading probability 
tables backward, attempting to find points on the curve where the 
relationships between sample size and difference-error ratios 
become linear. Of course, the above minimum limit is based on 
classical theory. Controlled laboratory experiments in which the 
researcher begins with a sample of one are also feasible. Once 
the researcher has found techniques which work for that one person, 
he can then increase his sample size sequentially, specifying 
which treatments are appropriate for which individuals under a 
given set of conditions. 

For factor analytic studies, we strive for minimum samples of 
around one hundred simply because experience indicates that the 
raw data coefficients of factor analyzing begin to stabilize with 
this size of n. 



REPRESENTATIVENESS 

What is done in survey work, especially in practice, differs 
greatly from what should be done based on the large body of avail- 
able empirical data about good survey methods. As we know, in 
initial broadcasting of a mail survey, approximately 50% of the 
individuals will respond. Subsequent follow-ups will pick up 
more individuals who tend to be different in nature from those 
gathered in by the first request. Generalizations based on re- 
sponses from less than 80% of the sample are extremely tenuous. 
Results based on a carefully canvassed sample of 100 individuals 
with effort being spent to insure almost total response may be 
much more meaningful than those based on responses of 5,000 indi- 
viduals who represent only a 50% return of the original selection. 
In both cases, the cost might be almost the same, but the value 
of the information from the smaller but high return sample is 
clearly more valuable. 

A common dodge to circumvent the problem of low-sampling 
returns is to select a group of individuals to sample and at the 
same time to randomly select an alternate for each person in the 
chosen sample. What the researcher fails to realize is that his 
sample now must be considered to be all individuals who responded 
as well as all individuals who did not respond. The probabilities 
are high that the researcher has merely increased the representa- 
tiveness of the number of cooperative respondees. Let me clarify. 
Researcher "x" selects a sample of 50, along with 50 alternatives. 
Twenty-five people respond on first contact. He uses 25 alter- 
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natives, 12 of whom respond. His sample remains at a 50% return 
because his total sample was 75, not 50; and he has obtained 
responses from approximately half the individuals. He would better 
have spent his time persuading the reluctant individuals to respond, 
or trying to determine why they were reluctant to respond. 



ADEQUACY OF CONTROLS 

Selecting adequate controls is a major bug-a-boo for educa- 
tional researchers. A common mistake is to choose school "x" as 
experimental and school "y" as control. The error in this approach 
is that there may be intrinsic differences between the students or 
environment in schools that account for the observed group dif- 
ferences other than those attributed to the experiment. (Stanley 
and Beeman, 1958, p. 94.) The final effect of such restrictions 
is that most studies have to be done within schools which have 
two or more classes of any one age level mee^;:ing at any one time, 
allowing the students to be randomly assigned to control "x" or 
treatment "y." One can better generalize, however, to a defined 
population in this latter situation. 



ABSOLUTE VERSUS RELATIVE CONTROLS 

Control problems are further complicated by the fact that in 
most educational situations it is impossible to use an absolute 
control method of completely withholding treatment. Even if the 
researcher does withhold treatment, he cannot assume that nothing 
is happening to the individual from the initiation to the com- 
pletion of the study. For example, in studies of psychotherapy 
control groups where supposedly treatment was withheld, there is 
indication that such was hardly the case at all. (Orne, 1962) 
Investigations of the control groups indicated that these indi- 
viduals talked about their problems to friends, ministers, and 
their physicians. If a person is hurt enough to need help, he 
will find it. If he is not hurt enough to need help, he may not 
be comparable in make-up to the experimental treatment groups 
at the initial phase of the study anyway. 

In educational research, relative controls ("competing 
treatments") are usually used in which the researcher attempts to 
answer the question of whether treatment "x" is better than "y"; 
not whether either "x" or "y" are of any value at all. With 
relative controls, when the researcher finds differences, he "wins" 
because he can interpret one method as superior to another; but 
when no differences are apparent, he "loses" because he is not 
sure whether the treatments were equally good or equally poor. 
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Educational researchers use matching as an attempt to develop 
adequate controls. That is, they build their experimental and 
control groups by matching individuals on one or more traits which 
are known to correlate with the criteria. On first inspection, 
this approach would seem desirable because the researcher could 
be fairly certain his experimental and control groups were equal 
before beginning the study. However, matching has a number of 
pitfalls: 1) when subjects are matched on a number of variables, 

results tend to be conservative estimates of differences because 
extreme individuals are lost, leaving a restricted range in vari- 
ability; 2) the researcher is never quite sure which are the 
pertinent variables on which to do the matching; and 3) just about 
any variable one selects on which to match has an error of measure- 
ment which gives no real assurance that equal scores represent more 
than some similarity on the variable. 



REPLICATION— CROSS VALIDATION 

The most common inadequacy in educational research design is 
lack of evidence of replication or cross-validation, i.e. repeat- 
ing the study at least once. Most educational experimenters do 
their study once and then, depending on whether they get positive 
results, publish their findings. I have often wondered what would 
happen if we could adequately keep track of all the negative- 
findings studies. It has been said that our field needs a Journal 
of Experimental Failures or Negative Findings , When the "misses" 
are ignored, we are enamored by the "hits." Our probability 
estimates for the studies that do get published are thus not 
accurate representations of reality. Ultimately, we can only hope 
that journals will 1) reject studies which lack replication, 2) 
properly label such research as pilot or exploratory studies, or 
3) publish brief abstracts of failures. 

Poor operational definitions account for many of the problems 
we have in attempting to replicate each others work. For example, 
Payne and I (1964) found many different definitions of the term 
"under- and over-achievement." When we applied these definitions 
to the same data we found they selected different individuals. 

It is easy to see that generalizations based on the various 
definitions might be to entirely different populations, question- 
ing whether one should expect verification at all of the original 
study being replicated. 



PLACEBO EFFECTS 

It is well known that just doing something will produce change 
in some individuals. Such a principle is extremely important to 
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the educator because it may be the show of interest on the part 
of the experimenter in the subjects or the novelty of the experi- 
ment which produced the findings. The usual assumption is that 
selecting a control group will solve the problem, but this is not 
always the case. The fact that an experiment of any type is going 
on in the school setting may have subtle, pervasive effects. It 
is not uncommon to find the control group improving as much as 
the experimental group — in few cases, even more. 

One approach to coping with this problem when a teacher has 
both experimental and control groups in the same class is to 
select a "pseudo-control" group. That is, an experimental group 
and two control groups are selected. The teacher is told about 
the "pseudo-control" group and given instructions to treat them 
as she would "normally." What she does not know is that the real 
control group is another sample not revealed to her until the 
study is completed. 



REGRESSION EFFECTS 

Ever since human individual differences have been studied, 
it has been known that under repeated testing the extremes tend 
to regress toward the mean. Studies based on individuals who are 
selected on the basis of scoring at the extremes of an independent 
variable are particularly vulnerable to this effect. The forgetful 
researcher who overlooks this principle might find himself in a 
predicament where his high groups get worse and his low groups 
get better after exposure to experimental treatments. He scratches 
his head in bewilderment trying to interpret his findings until 
some bright graduate student points out the obvious. 



PRE- POST-TEST EFFECTS 

Designs which involve testing at the beginning of a study and 
then testing after treatment to measure change are the vogue. It 
would seem like a perfectly logical approach to research until one 
realizes that the pre-testing broadcasts what the experimenter 
anticipates as outcomes of the treatment. The use of a control 
group helps to balance the effects of the initial test on final 
outcomes, but does not solve all problems associated with this 
design. For example, the problem still exists of the notorious 
unreliability of "change" or "difference" scores. 



ADEQUACY OF CRITERIA 

Limitations of criteria beseige the educational researcher 
in many domains — particularly if he is attempting to measure 
personality traits or values. The educational researcher is not 
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free of the problem even when he assesses in the traditional 
academic achievement and aptitude areas which have a long history 
of developing acceptable measures. 



PRE- POST-TEST PROBLEMS 

Trying to measure a child's increment of improvement in the 
achievement domain is difficult. Suppose the researcher uses the 
same test at the beginning and the end of the study. He then 
faces the problem of finding a test which will be sufficiently 
easy to permit the student to be on the scale at the beginning 
of the study and still be difficult enough so that there is ceiling 
left for improvement at the end of the study. Equivalent forms of 
the same test can ease this problem but not completely eradicate 
it. 



OUTCOME BIASING 

When a researcher uses a test to measure the outcomes of an 
experiment, he has to be extremely careful that he has not pre- 
judiced his results by choosing a test which favors or disfavors 
one of the experimental treatments. One solution is to involve 
disciples of each treatment in the selection of a fair criterion. 

The training of the experimenter also contributes to a source 
of error in educational experiments. Human beings being what they 
are, apparently they are not quite able to mask their hidden pref- 
rences for one method over another. We have learned to monitor 
our experimental treatments to attempt to determine if the treat- 
ment were actually administered. But some research questions how 
effective we are, even with these sanctions. (Orne, 1962) 

Surveys are particularly subject to biasing difficulty. One 
has only to examine the myriad of questionnaires sent out each 
year in the educational field to observe that the conclusions for 
some of these studies could be written before the data was re- 
turned. Removing biases is extremely difficult. Partly it is a 
matter of good sense; pilot testing of surveys on unsophisticated 
individuals with close monitoring such as asking for explanations 
for each of their responses helps. 



VALIDITY PROBLEMS 

A frequent error that researchers in education make is to 
attempt to evaluate the outcomes of a study using a single crit- 
erion. The problem becomes further compounded when the criterion 
has only questionable validity. In the past few years, thanks to 
the work of measurement people, we have developed much clearer 
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standards of what constitutes a valid measure. For research pur- 
poses, we accept devices with restricted evidence of validity but 
when we do, obviously we increase the probability of negative or 
uninterpretable findings. 



RELIABILITY 

We are beyond the point where we mistake stability as evid- 
ence of validity, but we still ignore the impact that an unreli- 
able measure can have on making probability statements. Unstable 
measures increase the error of our estimates and probably contri- 
bute to conservative estimates of true differences. 

We are often insensitive to reliability problems in the 
criteria we are trying to predict. If we are doing an experiment 
in which we are trying to predict a child's grade point average, 
we have to appraise the effect of the unreliability of the judg- 
ments of the teachers who provide the composite estimate of the 
child's achievement. Some researchers rely on standardized 
achievement tests as criteria because they tend to have higher 
stability than teacher grades. However, this solution is not a 
panacea because some students do well in one situation and not in 
the other. The choice depends on what the researcher is interested 
in ultimately predicting. Designers of educational research need 
to provide evidence of the reliability of their measures with 
samples under study, for only in this way can we obtain some small 
estimate of the current stability of the criteria. 



ETHICS 

Realistically, the designer of educational research is not 
restricted not only to the formal or theoretical problems of 
design in conducting his inquiry. He must cope with the human 
concerns of the subjects of his experimentation and the members 
of the community where the research is being conducted. Some 
individuals have raised considerable flack on this issue to the 
point of having brought some projects to a complete halt. Recent 
experiences in medical research reported in the Saturday Review 
testify to problems of values in experimental research. The 
American Rsyohologist (1965) has devoted a whole issue to the 
problems of ethics of psychological testing. I can testify that 
the effects of the small, but vocal protestors are frustrating, 
particularly when timid government agencies force one to remove 
important questions which might offend the well organized opposers 
of human research. The State of Michigan is currently considering 
a law to prohibit the use of psychological tests without written 
parental consent, which would seriously hamper educational research 
efforts. We are partially responsible for the criticisms. If the 
researcher ignores the audience responding to the questions, he is 
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asking for trouble. He creates problems for generous school 
personnel who have agreed to participate in the study. On the 
other hand, a small vocal group should not make us "gutless" in 
our attempts to understand and help human beings. Children and 
adults are not so fragile and naive that questions about sex, 
religion, or personal values will in some sinister manner flip 
them into mental ill health. The researcher has the responsi- 
bility to 1) protect the confidentiality of the data 2) not put 
the individual results in the hands of untrained school personnel 
and 3) not release individual data from partially validated tests. 
Furthermore, as Carl Thoresen said at the 1965 meeting of the 
American Personnel and Guidance Association, we have the respon- 
sibility to seek the best treatments we can as part of our ethics. 
(Thoresen, 1965) 

The research designer must question whether he has the right 
to withhold from the child what is thought to be the best treat- 
ment available. Researchers who are conscientious on this point 
try to write into their design assurance that remedial help will 
be offered to any subjects who are handicapped because of the 
inadequacy of a particular treatment. 



FUNDS FOR EDUCATIONAL RESEARCH 

One of the problems that a designer of educational studies 
faces is trying to find sources of funds to support his endeavors, 
particularly if he wants to do basic research. It is easy to 
understand why this situation comes about after one works with 
school administrators. The whole reward system is set up on 
almost the antithesis of how we do research. The school adminis- 
trator is judged by his effectiveness, which generally translates 
to being his capacity to almost simultaneously sense a problem 
and take action. Over the years, many administrators have learned 
that for the school bond issues, this approach can lead to disaster. 
Here he tries to analyze the problem by using community surveys, 
identify the hard and soft change elements, set up a plan of 
action in light of the mailable variables and then steadfastly 
work to attain the goal which is measured in the sought outcome — 
a favorable vote. But it has taken years for some school adminis- 
trators to arrive at this point with this one element of their job, 
and for some it has been an extremely painful learning experience. 
The problem of educational researchers comes in the fact that 
these very men who are so prone to immediate action with low 
tolerance for ambiguity, migrate to the controlling boards or 
administrative positions of foundation granting agencies. One 
year applied studies are welcomed; two year studies are tolerated; 
three year studies are looked upon with askance; and studies 
requiring more time than this are obviously subversive. In the 
past, the U. S. Office of Education has been a likely source to 
turn to for support of basic educational research — both in the 
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training of researchers and in finding projects. The recent 
shift in policy in the U. S, Office of Education indicates that 
those days may have come to an end. The dialogue between the 
respected professionals of the educational field and the members 
of the U. S. Office of Education is being severed. 



In recent years, educators have received considerable help 
from the physicists, biologists, mathmeticians, and chemists in 
curriculum development. Now we are going to need the same support 
in loosening the private foundation for governmental agency sup- 
port for basic educational research. 



A PARTING COMMENT 

We who do educational research have many assets in doing our 
work that are not fully realized by outsiders and sometimes by 
ourselves. School administrators may not think the same way we 
do about problem solving but they have an excellent record in 
opening their schools to our pursuits; a recent survey (Clasen, 
et. al. , 1966) substantiates this point and further indicates 
that the door to access is some form of a personal contact. In 
highlighting our problem, I do not want to indicate that the 
situation is totally bad, or without many rewarding features. 
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A Strategy for Psychological Research 
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The basic proposition underlying this paper is that no single 
theory or method is adequate for the scientific study of man and 
his environment. A general strategy for psychological research 
that transcends specific theories and methods is proposed. The 
concept of "ideal types" serves as the unifying theme for the 
proposed strategy. Included in the strategy are three basic 
steps: 1) investigation of the present situation, 2) construction 

of an ideal type, and 3) formulation and implementation of recom- 
mendations. Within the proposed framework, the traditional meth- 
ods of empiricism, rationalism, and pragmatism may be viewed as 
being complementary rather than contradictory. 



INTRODUCTION 



Psychological theory is currently plagued with controversy. 
There are Skinnerians and there are Rogerians; there are Freu- 
dians and there are neo-Freudians; there are Gestaltists, pheno- 
menologists, functionalists, existentialists, field theorists, 
information theorists, pragmatists, neo-pragmatists, and others. 
Many of these psychologists seem to believe that their school or 
theory is right and that all others are wrong. Often the contend- 
ing parties argue for their theory within the framework of that 
theory. Few psychologists are able to "move back" from their 
preferred theoretical orientation in order to compare and contrast 
two or more theories objectively. Confusion, uncertainty, and 
conflict are evident throughout psychology. 

Associated with this conflict among theoreticians in psychol- 
ogy is the ongoing debate about methodology. The clinical method 
and the statistical method, for example, are frequently compared 
(see Meehl, 1954) . The advocates of the clinical method believe 
that they obtain a more valid picture of man, while the propo- 
nents of the statistical method believe that their approach is 
more scientific. And so the argument continues. 
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The basic proposition underlying the present paper is that 
no single theory or method is adequate for the scientific study 
of man and his environment. A corollary to this proposition is 
that each theory and each method may be appropriate for certain 
problems. It then follows that there is a need for a framework 
or rationale for comparing different theories and different 
methods, and for selecting the ones that are most appropriate for 
given problems. 

The purpose of the present paper is to propose a general 
strategy for psychological research that transcends specific 
theories and methods , but makes use of all of them. The concept 
of "ideal types" serves as the unifying theme for the proposed 
strategy. 



PRINCIPLES 

Max Weber introduced the notion of "ideal types" in the 
social sciences. Weber clearly demonstrated the usefulness of 
comparing actual events in history with "ideal" constructions of 
these events (Gerth and Mills, 1958; Parsons, 1964; Weber, 1949). 
More recently, the concept of ideal types has been used in the 
behavioral sciences. Tanner, Birdsall, and Clarke, for example, 
have shown the value of formulating an "ideal observer" in psy- 
chophysics (1960) . Similarly, Anatol Rapoport and others have 
discussed the pros and cons of constructing "rational strategists" 
in the study of conflict and cooperation (1964) . 

In each of these uses of ideal types, some type of comparison 
is made between the empirical and the rational, between the 
realistic and the idealistic, or between the descriptive and the 
prescriptive. It is not one versus the other but the relation 
between them that provides the basis for understanding complex 
situations involving man and his environment. Watkins suggests 
that "One might improve one's appreciation of a roughly circular 
object by placing over it an accurate tracing of a circle" (1953, 
p. 725) . 

The impetus for the current use of ideal types results from 
studies in operations research, systems analysis, and cybernetics. 
Many of the specific studies in these relatively new disciplines 
apply some version of the following three-step strategy; 

1) Investigation of the present situation, 

2) Conceptualization of a desired or "ideal" situation, and 

3) Formulation of recommendations for moving from the actual 
toward the desired. 

The literature offers several different definitions of ideal 
types. Lefebre defines ideal types as "conceptual constructions 



o 



16 



X 




which portray what would be possible if all possibilities of a 
given set of conditions were completely carried out" (1957, p. 490). 
Watkins suggests that "A holistic ideal type is not a guess about 
reality, but an a priori word-picture — in other words, a defini- 
tion" (1953, p. 726) . Talcott Parsons points out that the ideal 
type as used by Max Weber is both abstract and general: "It does 

not describe a concrete course of action, but a normatively ideal 
course, assuming certain ends and modes of normative orientation 
as 'binding' on the actors" (1964, p, 13). 

For purposes of this paper, an ideal type is defined as a 
conception of the optimum state of a given system. A system may 
be defined as an assemblage of elements or variables united by 
some form of regular interaction or interdependence. The state 
of a system is the set of numerical values that its variables 
have at a given instant in time. An optimum state of a system 
is the most favorable state according to specified criteria. A 
system in psychology might be a set of behaviors, a person, the 
decision-making mechanism of a person, or even a body of knowledge 
in a given area. In regard to the proposed strategy, psychologists 
might formulate ideal types for any of these systems. 

The term "ideal" is frequently associated with words such as 
"excellence" and "perfection." It must be noted, however, that 
the values associated with ideal types, as the term is used here, 
may not correspond to the values of society. Evaluative adjec- 
tives, such as "good" or "bad," are meaningful only insofar as 
they relate to the arbitrary criteria established by the investi- 
gator. Thus, a psychologist might just as easily construct an 
ideal type of a criminal as an ideal type of a policeman. In 
either case, the ideal is merely a conceptualization of the best 
possible "performer" according to certain arbitrary criteria. 

It is also important to note that an ideal type, as used 
here, is not illusional. Ideal types go beyond actual knowledge 
of the present situation but are influenced by practical con- 
siderations. "Ideals are admittedly not fully realizable," 
according to Michael Polanyi, "but they must not be wholly 
impracticable. Their status is like that of 'pure engineering,' 
which I have defined as comprising the operational principles of 
machines" (1963, p. 63). Thus, an ideal type may be viewed as 
a conception of a desired situation that is grounded in reality. 

The proposed strategy provides a framework for answering 
these three questions: i) What is the present situation? 2) What 

would be an ideal situation? and 3) How can the ideal be achieved? 
Guidelines for answering these three questions are presented be- 
low. 

To answer the question "What is the present situation?" the 
psychologist needs a comprehensive understanding of the present 
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situation. He attempts to view the problem situation in the 
broadest practical context and explores each facet of it from 
different points of view. He formulates basic questions that 
need to be answered about the problem situation. The psychologist 
searches for counter arguments to his primary arguments and at- 
tempts to identify inconsistencies in the situation. He effec- 
tively applies the dialectical method of thesis, antithesis, and 
then synthesis, in order to get every possible insight into the 
present situation. Data are collected and analyzed as needed. 

Guidelines for answering the question "What would be an ideal 
situation?" show what might b^ accomplished. The psychologist 
must specify arbitrary criteria for the situation, such as max- 
imum performance, minimum cost, and minimum risk. He also must 
identify the actual constraints imposed upon the situation, such 
as time, money, and the innate capacity of people. Then, within 
the framework of the arbitrary criteria and the actual constraints, 
he generates new concepts that go beyond the present state of the 
system under study. These concepts may be developed by brain- 
storming, question asking, systematic ideation, or any other 
method that is conducive to the generation of new ideas. 



Guidelines for answering the question "How can the ideal be 
achieved?" are directed toward identifying the reasons for dis- 
crepancies between the actual and the ideal, and recommending 
courses of action for moving from the former toward the latter. 

The psychologist first attenpts to identify the obstacles stand- 
ing between the actual situation and the accomplishment of the 
ideal. He considers all major avenues for moving from the actual 
toward the ideal, which includes the formulation of "extremes" — 
the worst possible alternatives and the best possible alternatives. 
He "thinks through" the likely consequences of various alterna- 
tives, and he demonstrates the reasonableness of certain alter- 
natives and the futility of others. He then takes appropriate 
action. 

To apply the above guidelines effectively, the psychologist 
must have a comprehensive view of his field. This paper proposes 
a general orientation that transcends specific theories and 
methods, but makes use of all of them. The essence of this idea 
is illustrated in Figure 1, in which the field of psychology is 
conceptualized in terms of a three-dimensional figure, with 
problem areas , theoretical frameworks , and methods representing 
the three axes, and findings representing the content of the 
cube. These terms are defined below. 

A problem area is the subject of interest to the psycholo- 
gist. This might be learning, intelligence, motivation, person- 
ality, communication, or decision making. The problem area is 
the starting point for the inquiry. 
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RELATION AMONG PROBLEM AREAS, THEORETICAL 
FRAMEWORKS, METHODS, AND FINDINGS 



19 



A theoretical frciinework may be defined as the general 
orientation of the investigator; it is his subjective point of 
view. Examples of theoretical frameworks are common-sense psy- 
chology, existentialism, dynamic psychology, psychobiology, 
behaviorism, and cybernetics. Each of these takes a somewhat 
different view of man, and each emphasizes certain aspects of 
man. Theoretical frameworks cannot be proved or disproved; 
they are not true or false. Some, however, may be more useful 
than others. 

Methods represent different "ways of knowing" for the psy- 
chologist. Examples of methods are: everyday observation, 

introspection, case study, field study, laboratory experiments, 
and models. These different methods provide different types of 
information about man and his environment. No single method may 
be viewed as the "final word;" each has its own strengths and 
limitations. As we cross the gamut from everyday observation to 
models, for example, we move from little control to considerable 
control. At the same time, however, we also move from realistic 
human behavior and realistic conditions toward simulated behavior 
and simulated conditions. The investigator obviously must con- 
sider the "trade-offs" for each particular situation. 

The findings referred to inside the cube in Figure 1 are 
the results of combining a particular theoretical framework and 
a particular method to investigate a specific problem area. A 
well-conducted study should produce useful information regardless 
of the underlying theoretical framework or the method used. For 
example, data collected through everyday observation are obvious- 
ly different from those collected in a controlled laboratory 
experiment. This does not mean that one set of data is of less 
value than the other, but merely that they must be interpreted 
differently. "In the language of science," according to Bron- 
owski , "every fact is a field — a crisscross of implications, 
those that lead to it and those that lead from it" (1965, p. 52). 

The proposed approach encourages the psychologist to view 
his particular field of interest from "outside" the cube shown 
in Figure 1. Frequently, a psychologist attempts to structure 
his entire field of specialization in terms of a single theoret- 
ical framework or a single method. This can lead only to bias. 
The proposed approach gives the psychologist access to the entire 
cube. Jaspers suggests that, "A wealth of well-established 
viewpoints and an appropriate adaptation of these to the individ- 
ual case marks the ideal investigator" (1963, p. 825) . 

But here there is an obvious question: If the psychologist 

is able to assume a position "outside" the cube in Figure 1, what 
criteria does he use in assessing his approach? It would hardly 
seem reasonable if he used the same criteria as those imposed by 
one particular theoretical framework, say, behaviorism, or even 
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by one of the methods, say, laboratory experiment. Moreover, it 
would be just as unreasonable to establish a potpourri of un- 
related criteria extracted from all of the theoretical frameworks 
and methods. Thus, the need for a "higher authority" is apparent. 

It is proposed that reason is the most meaningful "higher 
authority" for the psychologist. Reason, while being difficult 
to define, does have certain distinguishing characteristics. It 
is man's highest power of intellect. Reason is a joining of logic 
and intuition, grounded in facts but inspired by imagination. It 
unites reality and possibility by establishing a fact, extra- 
polating beyond this fact, and then establishing another fact. 

It relies on rules of logic, but is more than these rules of logic. 
The presence of reason can lead to scientific discourse among all 
psychologists — from existentialists to behaviorists , or from 
phenomenologists to mathematical-model builders. Reason provides 
the common ground for all scientists. It is the essence of the 
proposed strategy for psychological research. 



PROPOSALS 

On the basis of what has been said thus far, a general strat- 
egy for psychological research is proposed. The three major steps 
included in the strategy are shown in Figure 2. 

In Step I, the present situation is viewed in the broadest 
practical context. The investigator attempts to get every pos- 
sible insight into the problem by looking at it in terms of 
different theoretical frameworks and by means of different meth- 
ods. Meaningful questions are generated. Data are collected and 
analyzed. 

In Step II, arbitrary criteria are specified, actual con- 
straints are considered, and new concepts are introduced. Then 
the investigator constructs a description of the most favorable 
state of the system under study. This description constitutes 
the "ideal type." 

In Step III, the psychologist formulates and implements 
recommendations. The relation between the actual and the ideal 
is analyzed, and an attempt is made to determine the reasons 
for any existing discrepancies. Different routes for moving 
from the actual toward the ideal are considered, and their con- 
sequences are evaluated. Recommendations are formulated and 
appropriate action is then taken. 

AN EXAMPLE 

A comprehensive study of human decision-making is used as an 
illustration of the application of the proposed strategy. The 
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FIGURE 2. MAJOR STEPS IN THE PROPOSED STRATEGY 
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psychologist first wants to attain a thorough understanding of 
empirical knowledge about decision-making. To achieve this under- 
standing, he constructs a framework consisting of problem areas, 
theoretical frameworks, methods, and findings, such as that shown 
in Figure 3 . 

By viewing the different problem dimensions "through the 
eyes" of different theoretical frameworks, the investigator is 
able to pose a number of questions. For example: 1) Common- 

sense psychology: What does my own judgment, based upon a life- 

time of everyday experience, tell me about decision-making? 

2) Existentialism: What is the basis for subjective probabil- 
ities? 3) Dynamic psychology: What motivates the individual to 

make the choice he does? 4) Psychobiology: What is the nature 

of decision-making behavior in brain-damaged patients? 5) Be- 
haviorism: What is the nature of the relation between contin- 

gencies of reinforcement and decision-making behavior? 6) Cyber- 
netics : What is the nature of the feedback loop in decision- 

making? 

Then, by use of a variety of methods, the psychologist pro- 
ceeds to collect information about human decision-making. For 
example: 1) he may observe children choosing team members for 

a baseball game; 2) he may introspect about his own decision- 
making experiences and attempt to identify significant factors 
that influenced particular decisions; 3) he may conduct a case 
study of a great leader considered to be an outstanding decision- 
maker; 4) he may investigate the relation between patterns of 
communication and the effectiveness of management decision-making 
in an industrial organization; 5) he may conduct a laboratory 
experiment to analyze the relative effectiveness of different 
types of reinforcement on decision-making; 6) he may study the 
decision-making behavior of an electronic computer programmed 
to simulate human decision-making behavior. 

After making a thorough analysis of his findings, the in- 
vestigator constructs a description of an ideal decision-maker. 

The arbitrary criterion in this particular case might be that 
the ideal decision-maker is able to select the best alternative 
from among those immediately available or potentially available. 
("Best" means that the value of the potential payoff of a given 
alternative is considered in conjunction with such constraints 
as time, effort, and risk involved.) The ideal decision-maker 
might then be described as follows: He has an effective over- 

all philosophy that guides his daily actions. This person does 
not assume constraints to exist when they actually do not. He 
is aware of the real alternatives in a situation — he is able to 
see below the surface. The ideal decision-maker is effective in 
coping with conflict situations, and he is able to think in terms 
of continue and probabilities. Alternatives are critically re- 
evaluated during the pre-decision period, which reduces regret 
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FIGURE 3. RELATION AMONG PROBLEM AREAS, THEORETICAL 
FRAMEWORKS, METHODS, AND FINDINGS IN 
DECISION MAKING 
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after the decision is actually made. He can deal effectively 
with both objective probabilities and subjective probabilities — 
and both simultaneously. He learns from his decisions and uses 
this knowledge in subsequent decisions. This is a description 
of an "ideal." 

The investigator studies the reasons for the discrepancies 
between the actual and the ideal, and then formulates recommen- 
dations. Some of the most general recommendations might be: 

1) researchers need to achieve a better understanding of the 
decision-making process and to develop methods for teaching 
people how to be better decision-makers; 2) the schools should 
consider training youngsters to be effective decision-makers in 
the primary grades; 3) the individual needs to formulate a 
meaningful over-all philosophy in order to be effective in his 
day-to-day decisions. The psychologist would also participate 
in the actual implementation of these recommendations. 

The three steps in the proposed strategy are obviously in- 
terrelated. The implementation of recommendations would be ex- 
pected to change the present state of the art in human decision- 
making, which, in turn, would bring about a new version of the 
ideal type, which would then lead to new recommendations, and so 
forth. We can describe the three steps in the proposed strategy 
as three interrelated components of a dynamic feedback-control 
system, in which each component continually changes its form as 
a result of the influence of the other components. 



A SUMMING UP 

A general strategy for psychological research has been pro- 
posed. The strategy is based upon the notion of an "ideal type," 
which is defined as a conception of the optimum state of a given 
system. The proposed strategy consists of three basic steps; 

I) investigation of the present situation, II) construction of an 
ideal type, and III) formulation and implementation of recommen- 
dations . 

There are several important characteristics of the proposed 
strategy. First, it assumes that the psychologist is interested 
in understanding and influencing man and his environment. Se- 
cond, it is problem oriented rather than theory or method ori- 
ented. Third, the strategy makes use of different theories and 
different methods, and does so in a systematic manner. Fourth, 
while it is grounded in reality, it encourages the behavioral 
scientist to look beyond what is presently known. Fifth, it is 
guided by reason. 
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The proposed strategy is related to other methods of in- 
quiry. For example. Step I, investigation of the present situa- 
tion, is empiricism in the best sense of the word. Step II, con- 
struction of an ideal type, may be viewed as one version of 
rationalism. Step III, formulation and implementation of recom- 
mendations, is clearly consistent with the pragmatism of William 
James (1907) . Each of these approaches — empiricism, rationalism, 
and pragmatism — offers something different to the behavioral 
scientist. Within the framework of the proposed strategy, however, 
these different approaches may be viewed as complementary rather 
than contradictory. 

Several words of caution are in order at this time. First, 
the proposed strategy encompasses both fact and speculation, and 
the investigator must keep in mind which is which. Second, we 
must not lose sight of the fact that ideal types are dynamic and 
can be expected to change with time. Jaspers illustrates this 
point by stating that, "Ideals of man collapse, but they serve 
as a goal to his march forward. Ideals in a sense can be sche- 
mata of ideas, road signs" (1949, p. 62). A third word of cau- 
tion is offered by Max Weber, who emphasizes that, "The construc- 
tion of abstract ideal types recommends itself not as an end but 
as a means" (1949, p. 92). 

The proposed strategy holds these promises for psychology: 

1) it can help bring together scattered facts in an objective 
manner, 2) it can provide a framework for determining the ap- 
propriateness of different theories and methods for given prob- 
lems, and 3) it can indicate directions for appropriate action. 

In addition, the proposed strategy provides a systematic means 
for isolating facts, assumptions, arbitrary criteria, specula- 
tions, ends, and means. 

We may conclude with a quotation from Karl Jaspers that 
points up the value of ideal types : "... in order to grasp 

the reality, we must see the possibilities. In the present, a 
formulation of the possibilities is the area in which I gain cer- 
tainty concerning what I decide; without possibility, I have no 
freedom; without a vision of the possibilities, I act blindly; 
only a knowledge of the possibilities enables me to know what I 
am actually doing" (1964, p. 238). 
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Planning the Reseach Program 
(A Summary of Reports) 
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The systems approach to planning a research program is very 
effective. Research efforts are directed toward specific objec- 
tives when researchers are looking at their resources and con- 
straints. The boundary lines, the alternatives, the costs and 
benefits become factors in the research planning. 

A systems approach involves great emphasis on the inter- 
related aspects of the whole. Flow diagrams are a valuable as- 
sistance in seeing the relatedness of the various aspects. Thus 
the research plan can be looked at as a road map. The map offers 
a general outline of where you are going and how you plan to get 
there. The plan offers direction but does not inhibit you be- 
cause it should be flexible. The research plan does not have to 
stifle creativity. On the contrary, it should foster creativity 
because it allows certain freedom within the sphere of activity. 

The persons who provide the plan should be directly involved 
in carrying out this plan from initiation of the idea to final 
evaluation. One of the first stages of planning is the concep- 
tualization of a major theme which tends to pull things and 
people together, to serve as an umbrella for the activity. 

Within such a conceptual framework the questions basic to 
systems analysis can be more effectively answered. Does the 
research effort have realistic objectives and are they stated in 
operational terms? What are the limits of the resources and the 
time available? 

What kind of standards will be built into the objectives? 

If the effect is going well, how will it be known? How can a 
system of checks and balances be built into the system? Are the 
researchers interested in the number of publications, number of 
hours expended, public opinion? 



While the program is in progress, will efforts be made to 
capitalize upon new sources of money, time and people? Does it 
make any difference if the research effort becomes involved in 
another area or concern which varies from the original concern? 
What are the restrictions built into the design of the program? 

How will the alternative cost and benefits of the program 
be estimated? What substitutions are available as a result of 
modifications? How will the costs of such things as reputation, 
frustration, inconvenience enter into the plan? 

Who will be responsible for implementing the plan? Has 
proper sequencing been considered? Which activities should be 
completed first and which completed thereafter in a workable 
sequence? Does a feedback group exist wherein corrective action 
can be taken if the need arises? Does a plan exist for updating 
the plans, or modifying the plan in order to make it more realis* 
tic. Is it possible to change the emphasis of what is being done? 

Sequencing provides special problems for the researcher. 
Sequencing involves both idea generation and idea evaluation. 
Researchers are generally geared to problem solving, but do not 
know how to identify a problem. Problem identification requires 
real skill and most people are not trained for it. Generally 
things are seen as we want them to exist. Researchers must be 
trained to see what actually exists first. Then, if there is a 
discrepancy between what exists and what we desire, progress can 
be made toward what we want with scientific help. 

Working at the interrelationships of problems is also im- 
portant. In addition, people often think differently about the 
same problem. Two differing persons or institutions could rein- 
force each other, aid in costs and provide an informal system of 
checks and balances. 

It is important to determine the results required and to 
estimate the time needed to produce these results. Sometimes 
a research group must have results the first year. Other re- 
search does not require immediate results. A hard look must be 
taken at all capabilities. A few people are top notch research- 
ers but aren't worth a fig in designing in terms of needed out- 
comes . 



In planning for research these steps are suggested: 

1. Identification of the areas of needed research; 

2. Assignment of some sort of priority to the various 
research areas identified; 

3. Development of a matrix or matrices of research activi- 
ties; 
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4. Planning the next steps — what research needs to be done 
presently, who should do it, how it should be done, and 
where it should be done. 



The Development of a Matrix 
in Distributive Education Research 



WILLIAM W. FARQUHAR 
Michigan State University 

Most anything we do can be researched. The problem is to 
decide those things that are important enough to warrant our 
attention, energies, and monies. Most frequently, the decision 
is made by independent investigators pursuing their own particu- 
lar interests. In recent years, as government and industrial 
funds have become more and more available for research purposes, 
the rise of a) a major organizational schema whereby a plan is 
adapted and sub-parts are confiscated by individuals and institu- 
tions, b) cooperative research ventures where a project is con- 
ducted by multiple investigators and c) the rise of research 
institutes or regional centers where a major theme of investiga- 
tion is carried out by a particular group have all become a 
forceful part of the research scene. Our discussion will be con- 
cerned with the framework of research which might fit any of the 
above. 

This paper will be concerned with 1) developing a matrix for 
planning research in distributive education, 2) a discussion of 
various types of designs which might be employed to answer some 
of the questions proposed by the matrix, 3) some general state- 
ments about the flow of research, 4) a discussion of the fine art 
of proposal writing, and 5) some general comments about processes 
for monitoring research activities. 



A MATRIX OF RESEARCH DESIGN 

There are any number of schema available for planning major 
research projects. For example. Figure 1 is extracted from the 
paper by Frank Wellman. I His approach has some fairly useful 
organizational purposes for the counseling process, but I serious- 
ly doubt its direct applicability to the kind of problem proposed 
for distributive education. It is possible to move from a matrix 



*F. Wellman, The Assessment of Counseling Outcomes: a Conceptual 

Framework, unpublished mimeo-paper, January 1967, p. 30. 
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such as Wellman has proposed to a model which approximates a flow 
chart. In Figure 2 an example is presented for the research and 
development center for learning and re-education. 2 Many examples 
of this type of model can be found in the literature. It has 
many uses but is limited in bringing about higher level experi- 
mental studies. I would like to propose one (Figure 3) which 
has utility for many different settings and one which I believe 
can be adapted to the needs of distributive education groups 
directly. If you will inspect Figure 3, you will see that the 
vertical axis is concerned with the problem areas which your 
group will have to define as being worthy of research attention. 

It would help if these problem areas were arranged in a hierarchy 
of importance. (The hierarchy may not contain absolute ranking, 
but it would seem to me that you should be able to arrive at a 
somewhat broad judgment of which problems are most pressing and 
important.) The horizontal axis contains three levels of re- 
search: descriptive, predictive and experimental. Let me define 

these more clearly. 

Descriptive : This type of research is commonly referred to as 

survey and constitutes a good launching pad for finding out the 
conditions which actually exist. One can think of research at 
this level as counting or identifying the "x's." It is usually 
not too exciting, but it certainly is necessary for a field to 
have a broad base of description in order to be aware of its 
current status. 

Predictive : From here on we begin to identify those "x's" which 
will tell us when "y" will occur. That is, we become more time 
oriented and become concerned with the linkage effects of certain 
elements. In fact, one could say that the beginnings of a theory 
are possible at this point. 

Experimental : The third type of research is where we manipulate 

the "x's" to produce desired "y's." Generally, research done at 
this level requires a fair amount of sophistication and planning, 
careful scrutiny in execution, and detailed analysis to be certain 
that extraneous variables do not account for the outcomes. 



DESIGNS FOR MATRIX 3 

Campbell and Stanley^ have summarized the effects of various 
components on internal and external invalidity of experiments. 



2Newsletter, Research and Development Center for Learning and 
Re-education, Vo I . II, No. I. 

3d. T. Campbell and J. C. Stanley, Expevimentat and Quasi- Experi- 
mental Designs for Research, Rand McNally and Co., Chicago, 1963. 
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Fig. 2 

Detailed Program Plan of the R & D Center for Learning & Re-education 
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TYPES OF RESEARCH 

FIGURE 3. A MATRIX OF RESEARCH PLANNING 



Their definitions are summarized below. 

Internal Invalidity 

History ; other events occurring between a first and se- 
cond measurement in addition to the experimental treat- 
ment. 

Maturation ; biological or psychological processes which 
systematically vary with the passage of time per se, in- 
dependently of the experimental treatment or other events. 

Testing ; the effects of a pre-test on subsequent behavior. 

Instrumentation ; changes in the measuring instrument it- 
self, or in the observers or raters, which might account 
for an - £2 difference. 

Regression ; the universal tendency of extreme scores to 
be closer to the mean on a second test. 

Selection ; pre-treatment inequality of experimental groups. 

Experimental mortality ; the differential loss of persons 
from the treatment groups. 

Selection-Maturation Interaction ; the effect obtained 
when treatment groups have different maturation rates. 

External Invalidity 

Testing and X Interaction ; in which a pre-test increases 
or decreases individual sensitivity or responsiveness to 
the treatment and thus makes the experimental results for 
pre-tested individuals different from what they would have 
been without the pre-test. 

Selection and X Interaction ; Generally speaking, the 
greater amount of cooperation the experiment requires , the 
more it disrupts routine, and the higher the refusal rate, 
the greater will be the selection and X interaction. 

Reactive Arrangements ; whenever an experiment has obvious - 
ly artificial aspects we need to fear reactive arrange- 
ments . 

Multiple-X Interference ; is a problem whenever multiple 
treatments are applied to the same persons. 

In Tables 1, 2, and 3 from their recent publications can be 
seen the control of the three sources of invalidity in relation 
to a number of designs. 
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THE FLOW OF RESEARCH 



If we assume that the purpose of research is to make proba- 
bility statements about the nature of reality or to discover 
knowledge, we can begin to catch the cycling implication of re- 
search. Research always starts with a hunch that a certain set 
of conditions exist, which are not known or that certain events 
will predict other events, or that the manipulation of certain 
events will produce certain results. Researchers do not set out 
to prove negatives. Our goal is to establish positive findings. 
Therefore, we begin with either: 

Level I: a hunch (a kind of focusing or pre-theory) , 

Level II: an accumulation of past studies which raise some 

question marks , 

Level III: a conceptualized framework or theory which gen- 

erates testable hypotheses. 

It is important that the researcher recognize where his 
launching pad of inquisitiveness exists because the three previous 
conditions determine the type of research hypothesis he will for- 
mulate. All have in common the fact that the research hypothesis 
anticipates a relationship or a difference to exist. However, at 
the lowest level (focusing, survey, descriptive) the researcher 
is often not able to designate what the direction or magnitude 
of the relationship will be. More highly developed research 
fields place more demand upon the researcher to anticipate a 
priori the direction of his hypothesis. 

Once the researcher has formulated his basic hypothesis 
which is in essence a narrowing of the definition of his problem, 
he then gathers data, manipulates that data in some manner and 
makes a decision about whether to accept or reject his hypothesis. 
In this process, his research hypothesis is changed to a statis- 
tical hypothesis. (A taxonomy of statistical tests for various 
designs and a number of statistical tests are included in appen- 
dix II.) 

For practical purposes, statistical hypotheses are always 
null in form, that is, we enter our probability tables with the 
assumption that no differences exist. (Mind you, this is simply 
a parsimonious, status quo, statistical maneuver, not a resume 
of our initial gut level plunge into doing research.) It is in 
the alternate hypothesis that we find the essence of the difference 
of the levels we employed when we initiated our studies. For 
example, if we have hypothesized only the null form, Hq : u^= U2 
the alternate hypothesis (Level X) not designated, we are pre- 
pared to accept differences no matter whether ui>U2 or ui<U2. 

This is what we call a two-tailed test. If we had moved to levels 
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II or III where we had past studies or theory, our hypothesis 
stated in the same initial null Hq : u^ = U 2 but in addition we 
designate either ui>U 2 or U 2 <ui. Because we will only accept 
one condition, we employ a one-tailed test which generally means 
that the ?*.05 will actually be equal to the table value of 9*. 10 
because most of our statistical tables are two-tailed. We now 
have three conditions to cope with: a) we find that Hq : u^ does 

equal U 2 b) Hq: ^ U 2 and that : ui>U 2 as we had hypothe- 

sized so our research hypothesis is confirmed or that c) Hq : u^;^ 
U 2 and that : Ui<U 2 which we did not hypothesize. In Cases 
1 and 3 our research hypothesis is not confirmed. In Case 1, we 
may decide to abandon the research for any number of reasons 
some being that the original idea was poor or the instrumentation 
was inadequate. In the second case, where we^ have a confirmation 
of our hypothesis, we should be able to tie our finding back to 
our theoretical structure and test other elements or other con- 
structs . In the third case where our findings were contrary to 
prediction, it is almost mandatory that we replicate the study 
but that prior to this we revamp our theoretical base for making 
our hypothesis. A point of caution, if we had done a number of 
hypotheses and only one or two came out contrary to prediction, 
the theory may stand but we may have to modify some of the con- 
cepts of how side variables influence outcomes, i.e., social 
expectancy. 



THE ART OF PROPOSAL WRITING 

Krathwohl has prepared a carefully documented guide for 
writing research proposals. ^ i have a few general comments 
about proposal writing. 

1. I think it is extremely important that you know the 
nature of the agency for whom you are preparing the 
proposal. This is the same advice one would give to an 
author in writing a book. 

2. If you can in any way have part of the research begun 
so that the funding agency is convinced of the serious- 
ness of your intentions the chances are increased that 
your proposal will be accepted. 

3. Be extremely cautious in the kinds of comments you make 
in the proposal because the agency will undoubtly hold 
you financially responsible for completing those things 
for which you have contracted, i.e., state samples of 
approximately such and such a number will be gathered. 



^D. Krathwohl, How to Prepare Research Proposals, unpublished 
paper. 
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4. Prepare your budget carefully with the help of an ex- 
perienced person. Your goal is to secure the proposal 
with sufficient funds to conduct the project, but with- 
out such a large outlay that the proposal will auto- 
matically be turned down as being too expensive. Remem- 
ber that no matter how close you figure your expenses , 
unexpected events will occur and they always cost money. 

5. Try to secure the grant with the line items freed so that 
you can shift them without long entangled red tape. This 
will give you much more flexibility in conducting your 
research. 

6. Remember that the bureaucratic mind operates in such a 
manner that he wants justification for your expenditures. 
Consider travel for example. You probably will not 
know all the travel you will need to do, but you can make 
average estimates (and then add a few because you will 
always have to go back — particularly when you are trying 
to secure cooperation in a project) . 

7. Be cautious of half-time secretary commitments. It is 
hard to secure a part-time person. 

8. Choose your co-workers carefully. Supplement your own 
limitations, i.e., if you are compulsive, find a more 
expansive free-wheeler to lend balance. (But make sure 
he will come through on the work.) 



MONITORING RESEARCH ACTIVITIES 

The PERT process of monitoring research activities has un- 
usual value in coordinating long-range planning. Desmond Cook's 
work on this topic is well known. In brief, it is a system per- 
mitting the individual to map his critical decision-making points 
— thus permitting preparation for those points and estimates of 
time needed to get from decision to decision, or from initiation 
to termination of the project. 
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INTRODUCTION 

The need for a program of research in distributive education 
was indicated at a conference held in Washington, D, C, in June 
1960. One of the findings and conclusions reported at this con- 
ference was the following: 

Distributive education in reaching maturity finds 
itself in need of principles against which, practices 
may be sounded. During recent years considerable 
emphasis has been given to questioning educational 
practices as well as practices in the field of distri- 
bution, Distributive educators have been in the un- 
happy position of having very little literature in their 
field. Information about the majority of areas basic 
to the development of the distributive education pro- 
gram has not been available. Such provincialism must 
be eliminated. 

There is tremendous need to find answers and to 
find ways to get at the answers. Distributive educa- 
tion has arrived at a period in its existence when it 
is important to locate significant opinion and erroneous 
opinion. The urgency does not lie in digging up new 
facts or in conducting research simply in order to 
have research. In this bread-and-butter stage, the 
primary job is one that calls for careful thinking to 
identify and solve problems. 

The immediate goals of such an approach can be 
realized through a 4-step procedure. 






1. Identifying the areas in which facts are 
needed. 

2. Finding out who has conducted studies and 
what these studies have revealed in the areas 
in which facts are needed. 

3. Applying these findings not only to content 
but also to practices and methodology in 
distributive education. 

4. Conducting studies in the areas in which no 
answers are now available. 

Distributive educators must find out what research 
has been done or is going on that has implication for 
distributive education and then find out how this tests 
out in terms of practices and program operation. The 
importance of learning how to evaluate both formal and 
informal research needs to be recognized more adequate- 
ly. This is a responsibility of all distributive 
education personnel. 



TIMELINESS OF RESEARCH EMPHASIS 

In view of the status of the distributive education 
program, events taking place in the national economy 
and in the schools, plus the awakening interest of 
distributive educators, the present is a timely period 
for initiating nationwide emphasis on simple, formal, 
cooperative, and experimental research. The benefits 
of such emphases will be felt at all levels of program 
operation through the development of services factually 
conceived and confidently provided by distributive 
education leadership. 

Now is the time to create the authority and the 
foundation for growth of the distributive education 
program. Now is the time to prove distributive educa- 
tion's worth and to reinforce the acceptance and 
respect of those who are asked to provide funds and 
lend support to this program. Today distributive 
educators must find out what they need to know in 
order to develop the kind of program they want to 
have several years from now. The trend away from 
provincialism toward professionalism is evident. 
Distributive education can become a truly professional 
field through the quantity and quality of the research 
it makes available and uses in the development of its 
program. 
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In accepting research as the substance of dis- 
tributive education, it might seem logical to establish 
a broad base for such activities. Actually the con- 
trary should be true. Distributive education research 
should have a specific focus in order to bring to the 
total program a productive concentration of resources 
and people. I 

At this conference, considerable attention was also given to 
the establishment of research priorities. In this regard the 
following recommendation was made: 

In setting up the desired research program it 
would be important to determine research items for 
iitmiediate action and others which would be planned 
for a matter of five or ten years hence. These would 
be assigned to their proper level and research class- 
ification, perhaps by means of a checklist of minimum 
research to be carried on within each level of activity. 

This would encourage the designation of a wide range 
of studies as the preferred action program. The top 
priority at each level should be given to that research 
which would help distributive education personnel 
develop the research habit and the research point of 

view. 2 

Although this need for a long range program of research was 
recognized, little, if anything, was done to implement the 
recommendations made at the National Distributive Education 
Research Conference in I960. 

The purpose of the first part of this paper is to review 
briefly the research planning seminar and report the various 
factors considered in the planning seminar. The latter part of 
the paper describes and outlines a research matrix which was 
later developed to fulfill one of the objectives of the seminar. 

As a preliminary preparation to the seminar the participants 
reviewed several publications concerned with distributive educa- 
tion, research design, and research management. During the 
seminar a review was made of the systems approach and other re- 
search management techniques available to determine if any of these 
could be useful in planning and programming a system with practical 
applicability in educational research. 



^U. S. Department of Health, Education and Welfare, Patterns of 
Research in Distributive Education, Washington, D. C.: Govern- 

ment Printing Office, 1961, pp. 11-12. 

^Ibid , , p. 14. 
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During World War II the concepts of operations research and 
the systems approach to research management by scientists and 
military personnel were developed. In the 1950 *s, the I, E, 

Dupont De Nemours Company developed a network analysis technique 
called the Critical Path Method (CPM) and later the Bureau of 
Ordinance, U, S, Navy, designed and implemented the Program Eval- 
uation and Review Technique (PERT) • Today about 50 variations of 
formalized planning, scheduling and control techniques can be 
identified,^ 

This paper will focus upon a technique of planning and pro- 
gramming research that has been found effective in bio-chemical 
research. 

Fundamentally, planning based on network analysis techniques 
consists of breaking down a project or work effort into a number 
of elements, determining the logical relationships between the 
elements, depicting these relationships by means .of a network, 
and making time estimates for the completion of each job or activ- 
ity, 4 The use of CPM or similar approaches requires that the 
project or work effort to be planned have several essential 
characteristics : 5 

1, It must consist of a collection of activities which are 
well-defined and which, when completed, mark the end of 
the project, 

2, Activities may be started and stopped independently of 
each other, within a given sequence, thus eliminating 
continuous- flow process activities, 

3, Activities must be in order, in that performance must 
take place in a given sequence, 

4, Target dates for completion of these activities are 
usually specified. 



•^R, L, Martino, "Finding the Critical Path," Project Management 
and Control* New York: American Management Association, 1964, 

Vo I , I , p , 5, 

4r, L, Brown, Network Analysis : Its Use in Research Management * 

London: British Coal Utilization Research Association, 1965, 

Gazette No, 52, p, I , 

^F, K, Levey; G, L, Thompson; and J, D, W, Wiest, "Introduction 
to the Critical Path Method," in J, F, Muth and G, L, Thompson 
(eds,). Industrial Scheduling, p,3l. 
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In most instances all of the elements of a research effort 
cannot be as specifically identified and described with the degree 
of precision represented by the above criteria. 

In research programs, where the tasks to be performed include 
many unknown variables and parameters, and when many different 
individuals and agencies are included, it is not possible to 
estimate accurately the duration of the project. 

In research efforts, the concept of "ordered sequence" has 
meaning only as it refers to the logic of scientific aspects for 
pursuing one of several lines of research for the purpose of 
acquiring the information base to pursue additional lines of re- 
search or until sufficient information is derived to validate as 
fact or particular research assumption or to reach an objective. ^ 
Although the importance of time as a resource must not be forgotten 
in any type of effort, it seems doubtful that sharp and meaningful 
time estimates can be developed as to when certain research activ- 
ities are to be accomplished or objectives reached. 

Some of these factors described prevent the direct applica- 
tion of currently available planning and control techniques to 
research efforts in general and to educational research in 
particular. Nevertheless, some of the underlying concepts and 
philosophies inherent in these planning and control techniques 
can be utilized to construct a general framework for the planning 
and programming of some types of research efforts. 



WHY PLANNED AND PROGRAMMED RESEARCH 

The benefits of applying more formalized planning techniques 
in such research situations are generally similar to the results 
experienced in planning other types of programs. Some of the more 
significant benefits claimed by experienced researchers include: 
a) the provision of a framework for the selection (in some situa- 
tions forcing the selection) of goals, objectives, and sub-objec- 
tives and a weighing of their importance; b) provision of a means 
for the orderly integration of many program elements and the 
determination of interrelationships and interfaces; c) the pro- 
vision of a logic framework for the establishment of priorities 
and the determination of required resources, often in the face 
of competition for resources that are unlimited; d) the pre- 
scription of information, monitoring, and decision-making require- 
ments and of responsiblities for operations within the framework 
of an integrated effort; e) reminding more easily those concerned 



^H. Eisner, "A Generalized Network Approach to the Planning and 
Scheduling of a Research Project," Operations Research, 1962, 

Vo I . 10, pp . 115-125. 
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that a sense of urgency may be in order for reaching some objec- 
tives and goals, 7 



GENERAL STEPS IN PLANNING AND 
PROGRAMMING RESEARCH 

There are certain criteria that should be met by a formalized 
and structured planning technique if it is to be meaningful and 
useful in the research environment. Two areas are of fundamental 
concern; 1) the basis for constructing the general program model 
and its detailed contents; and 2) the basis for determining the 
.sequential order of events. 

The formulation of a general model, the identification and 
description of major elements, and further reduction of major 
elements into smaller segments (i,e,, projects, events, activities, 
etc,) must be based on research logic, the substance of the work 
to be performed, and the discipline involved, 8 

The sequential ordering of efforts for performance within 
the model should proceed from the determination of the logical 
relationships between elements or events. Since capability to 
accomplish every event is either not known or cannot be established 
with any meaningful degree of accuracy, program success or progress 
is evaluated on the basis of events being accomplished, rather 
than specified periods of time or pre-selected target dates. 



DEVELOPING A RESEARCH MATRIX FOR 
DISTRIBUTIVE EDUCATION 

A modification of the Convergence Technique was developed to 
utilize some of the general features of the systems and network 
approaches in the planning of research. 

Basically, the technique involves the determination of a 
series of elements which are relevant to the overall objective 



^Louis Carrese, and Carl G, Baker, The Convergence Technique--- 
A Method for the Planning and Programming of Research Efforts, 
Washington, D, C,: National Cancer Institute, Department of 

Health, Education, and Welfare, 1966, 

^Ibid, 
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and sequentially ordered on the basis of research logic, and 
graphically represented by a matrix, 9, 10 

The basic proposition of the Convergence Technique modified 
for our purposes may be stated as follows: 

If the logic used for the construction of the 
matrix represents a valid model of the content of the 
program to be conducted, and if the sequential ordering 
of the program elements is accomplished on the basis 
of this logic, then in reality, as research elements 
or cells are implemented within the matrix, the 
intermediate objectives of each step and phase will 
be achieved and the scope of the program will become 
narrower until all efforts converge on the end point 
which has been established as the over-all program 
goal. I I 



CONSTRUCTION OF THE MATRIX 

Within the framework of some general procedural rules for 
the development and construction of the program matrix, use of 
the technique requires the f emulation of a logic system judged 
to be valid for the achievement of specific objectives in the 
area of research being planned. 

The most important steps involved in this process are: 

1, The identification of broad areas of needed research. 

2, The establishment of priorities of research efforts. 

3, The selection and fomulation of the end goal of the 
program and a series of major intermediate objectives 
requisite to the achievement of the program goal. 



, G, Baker; L, M, Carrese; and F, J, Rausher, ”The Special 
Virus-Leukemia Program of the National Cancer Institute: 
Scientific Aspects and Program Logic,” Symposium on Some Recent 
Developments in Comparative Medicine, Proceedings of the 
Zoological Society of London, (In press), 

G, Zubrod; S, Schepartz; L, M, Carrese; and C, G, Baker, 
"The Cancer Chemotherapy Program," A report to the National 
Advisory Cancer Council, August 1965, four volumes, Vol, LV, 
Proposal for the National Cancer Institute Cancer Chemotherapy 
Program, (In preparation for publication), 

IlCarrese and Baker, op, cit. 
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4, The identification of the various elements, segments, 
sub-divisions and dimensions of the program or problem 
area. 

5, The development of a logical system to provide the frame- 
work for the delineation of sub-units as indicated by 
these previously identified elements and dimensions, the 
determination of the logical-sequential order in which 
research is to be performed, and the establishment of the 
interrelationships among the research elements. 

In the development of the suggested matrix the general 
technique including the steps listed above were followed. 



IDENTIFICATION OF AREAS OF 
NEEDED RESEARCH 

After a review of research and the literature relating to 
research in education and distributive education, the following 
areas of needed research were identified: 

Philosophy and objectives 

Curriculum development 

Student personnel services (guidance) 

Learning processes — teaching methods 
Educational programs 
Facilities and equipment 
Instructional media 
Administration and supervision 
Evaluation 
Teacher education 
Research 

Manpower needs — employment opportunities 

These suggested areas are not necessarily thought of as 
distinct units or divisions in terms of contents or activities, 
but separate groupings of logically related categories based on 
the judgment and experience of those involved in the seminar. 



FIRST PRIORITY— CURRICULUM DEVELOPMENT 

Perhaps the most important and difficult decision made at 
the seminar was the identification of a single area of highest 
priority. After considerable discussion and reviews of literature 
and upon the advice of those present at the seminar, curriculum 

development was judged to be the area of most critical research 
need. 
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DEVELOPING THE GENERAL SYSTEM 



Steps three, four, and five outlined in the construction 
process were all interrelated and the matrices were developed 
after consideration of the following elements: 

Levels of research 
Occupational levels 
Educational levels 
Steps in curriculum research 
The competencies within the D. E. curriculum 
Standard terminology for curriculum and instruction in 
distributive occupational classifications 

The initial matrix of distributive education curriculum 
research was developed with the following dimensions or para- 
meters: (See Chart I) 

Competencies of areas of instruction 
Occupational and educational levels 
Steps in curriculum research 

Curriculums for instruction in distributive education may be 
classified as basic job curriculums, career development job 
curriculums and specialist job curriculums. Each corresponds to 
a level of employment responsibility and is identified with the 
degree of competency needed in specific distributive employment, I 2 
The levels are offered to high school and post-high school 
students as well as adults. 

The substance of the distributive curriculum is identified 
with the competencies universally needed in distributive employ- 
ment, The subject matter is divided, therefore, into areas of 
instruction which correspond to these competencies, I ^ Five major 
categories of instruction are included in each curriculum in order 
to develop competencies in the following areas: 

Marketing 

Product or service technology 
Social skills 
Basic skills 

Distribution in the economy I ^ 



*^11, S, Office of Health, Education, and Welfare, Office of Educa- 
tion, D'istributive Education in the High School, Washington, D, C,i 
U, S, Government Printing Office, 1965, p, 15, 

I ^Ibid , , p , 2 I 

^^Ibid., p, 22, 
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COMPETENCIES 



LEVELS 

INSTITUTIONAL-OCCUPATIONAL 




CHART I 
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FIRST ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH 
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The generally accepted steps in curriculum development are: 

Formulation of objectives 

Organizing the learning experiences (content) 

Evaluation 

Thus, the completed first-order matrix of distributive educa- 
tion curriculum research shows the following parameters and their 
logical sub-divisions, (See Chart 2) 

Educational Levels 
High schools 
Post-high school 
Adult 

Occupational Levels 
Basic 

Career development 
Specialist 

The Distributive Competencies 
Marketing 
Technology 
Social skills 
Basic skills 
Economic 

Steps in Curriculum Development 
Objectives 
Content 
Evaluation 

This matrix is the general model containing the major elements 
referred to on page 48, Its ultimate completion or implementation 
to the over-all program goal is the convergence. 

The sub-division of the matrix into sub-matrices is accomp- 
lished through further reduction of the major elements into 
smaller segments, A conscientious effort was made to observe the 
logical sub-divisions of each element and the taxonomies or class- 
ifications as generally accepted by the discipline. 

The development of a sub-matrix involves the same steps and 
same types of decisions as the development of the original matrix, 
i,e,, the establishment of priorities, the identification of the 
sub-units and the determination of logical sequential orders of 
activities. 



The marketing competency was selected as the instructional 
area with the highest priority. Sometimes referred to as the 



53 



COMPETENCIES 



LEVELS 

INSTITUTIONAL-OCCUPATIONAL 




CHART 2 
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FIRST ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH 
AFFECTING LEVELS, COMPETENCIES AND DEVELOPMENTAL STEPS 



"discipline of distribution" I 5 it is the one competency that is 
unique to our field. 

The marketing competency has been classified into six func- 
tions : I 6 

Selling 

Sales promotion 

Buying 

Operations 

Market research 

Management 

A sample or suggested second order matrix of distributive 
education curriculum researched on the marketing competency is. 
developed with the six marketing functions as sub-divisions of 
the marketing parameter. (See Chart 3) 

The development of the third order matrix proceeded on the 
decision that the determination of objectives was the logical 
first step in the sequential ordering of curriculum research. 

Educational objectives have been classified by Bloom, 
Krathwohl, and Simpson into three domains: Cognitive, Affective 

and Psychomotor. 17, 18, 19 

This classification was accepted in the sample third order 
matrix involving objectives of the marketing competency. (See 
Chart 4) 



*^Edwln L. Nelson, "Bases for Curriculum Development in Distribu- 
tion," A paper presented at the National Clinic on Distributive 
Education, Washington, D. C., October 1963. 

I ^Ibid, 

l7Benjamin S. Bloom, Taxonomy of Educational Obj^otivesj, Handbook 
I: Cognitive Domain, New York: David McKay Co., Inc., 1956, 

p. 207. 

I ft 

David R. Krathwoh I , Taxonomy of Educational Obfectivesj Handbook 
II: Affective Domain, New York: David McKay Co., Inc., 1964, 

p. 196. 

'^Elizabeth Jane Simpson, The Classification of Educational Object- 
ives: Psychomotor Domain, Urbana, Illinois: University of 

I I I inois, 1966, p. 35. 
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MARKETING FUNCTIONS 
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LEVELS 

INSTITUTIONAL-OCCUPATIONAL 
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SECOND ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH 

ON THE MARKETING COMPETENCY 



MARKETING FUNCTIONS 



I 



LEVELS 

INSTITUTIONAL-OCCUPATIONAL 
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THIRD ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH 
ON OBJECTIVES IN THE MARKETING COMPETENCY 



A sample fourth order matrix was developed based on the 
selling function in general merchandise instructional program 
listed in the Standard Terminology for Instruction in Local and 
State School Systems ,’2-0 it is evident that in a broad field such 
as distribution there is diversity in these functions as they are 
performed in the various areas. In order to break down this 
function into more accomplishable research segments, the U. S. 
Office of Education Classification was used. The major considera- 
tions affecting this decision was its general acceptance by 
distributive educators. 

The sample fourth order matrix involves the general merchan- 
dise program and its interrelationships with the institutional- 
occupational levels, the three domains of objectives and the 
instructional programs in the U. S. Office of Education Classifi- 
cation. (See Chart 5) 

The fifth order matrix represents further refinements of the 
U. S. Office of Education Classification by observing their 
process of sub-dividing the classifications and is concerned with 
the various general merchandise sub-divisions. (See Chart 6) 

Further reductions are possible, e.g., the sub-dividing of 
the cognitive domain into its broad categories and even further 
into its various sub-categories. The decisions regarding the 
extent of these further reductions may be affected by such factors 
as time, resources, talents and facilities of the researchers. 
However, it is believed that at this stage a researcher could 
select any cube within the suggested matrix as an individual 
research project. 

One example of a research effort that would be a logical 
step in the proposed programmed approach has already emerged, a 
master's thesis by Oma Rebecca Hawkins. 2 1 This is an excellent 
example of the contribution of a single researcher within the 
framework of a long-range program of research and development. 



2 0 

U. S. Office of Education, Standard Terminology for Instruction 
in Local and State School Systems^ State Educational Records and 
Report Series* Handbook VI. Washington, D. C.: Government 

Printing Office, 1967, p. 670. 

^*0ma Rebecca Hawkins, "The Construction and Classification of 
Educational Objectives to Develop Selling Competencies Needed 
by Workers in the General Merchandise Category of Distribution," 
Master's thesis, Virginia Polytechnic Institute, 1968. (Un- 
pub I i shed ) 
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OFFICE EDUCATION CLASSIFICATIONS 
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LEVELS 

INSTITUTIONAL-OCCUPATIONAL 
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FOURTH ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH ON 
OBJECTIVES WITHIN THE SELLING FUNCTION BY THE OFFICE OF EDUCATION CLASSIFICATION 



GENERAL MERCHANDISE SUBDIVISION 



LEVELS 

INSTITUTIONAL-OCCUPATIONAL 
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FIFTH ORDER MATRIX OF DISTRIBUTIVE EDUCATION CURRICULUM RESEARCH ON 
OBJECTIVES WITHIN THE SELLING FUNCTION OF THE GENERAL MERCHANDISE INSTRUCTIONAL PROGRAM 



This process of matrix development represents the application 
of a technique in the planning and programming of research. 

Neither the technique nor the matrices are considered to be fully 
developed. Refinements and modifications will emerge with con- 
tinued applications. As mentioned earlier, this proposed approach 
represents a modification of a system that has proved effective 
and productive in biomedical research. Today in distributive 
education we have a collection of independent researchers and a 
system of research and development centers, regional laboratories 
and research coordinating units. It is believed that the proposed 
matrix can be utilized by all of the above individuals and agencies 
in the development and implementation of a massive and significant 
research effort which will yield maximum returns to our profession, 
our students, and our economy. 
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APPENDIX 1 
SAMPLING CHART 



Type of 
Sampling 


Brief Description 


Advantages 


Di s ad van t age s 


A. Simple 


Assign to each 


1. Requires min- 


1. Does not make 


random 


population mem- 


imum know- 


use of know- 




her a unique 


ledge of pop- 


ledge of pop- 




number; select 


ulation in 


ulation which 




sample items by 


advance 


researcher may 




use of random 


2. Free of pos- 


have 




numbers 


sible class- 
ification er- 
rors 

3. Easy to ana- 


2. Larger errors 
for same sample 
size than in 
stratified 






lyze data and 
compute er- 
rors 


sampling 


B. System- 


Use natural or- 


1. If popula- 


1. If sampling in- 


atic 


dering or order 


tion is or- 


terval is re- 




population; se- 


dered with 


lated to a 




lect random 


respect to 


periodic or- 




starting point 


pertinent 


dering of the 




between 1 and 


property , 


population, in- 




the nearest in- 


gives strat- 


creased var- 




teger to the 


ification 


iability may 




sampling ration 


effect, and 


be introduced 




(N/n) ; select 


hence re- 


2. Estimates of 




items at inter- 


duces var- 


error likely 




val or nearest 


iability 


to be high 




integer to sam- 


compared to 


where there is 




pling ratio 


A 

2. Simplicity 
of drawing 
sample; easy 
to check 


stratification 

effect 


C. Multi- 


Use of a form of 


1. Sampling 


1. Errors likely 


stage 


random sampling 


lists, iden- 


to be larger 


random 


in each of the 


tification. 


than in A or B 




sampling stages 


and number- 


for same sam- 




where there are 


ing required 


ple size 




at least two 


only for 


2. Errors increase 




stages 


members of 
sampling 
units selec- 


as number of 
sampling units 
selected de- 






ted in sam- 
ple 


creases 
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APPENDIX 1 (continued) 



1. With 

probabil 
ity pro- 
portion- 
ate to 
size 



D, Strati- 
fied 

1. Propor- 
tionate 



2 . Optimum 
alloca- 
tion 



Select sampling 
units with 
probability 
proportionate 
to their size 



Select from every 
sampling unit 
at other than 
last stage a 
random sample 
proportionate to 
size of sampling 
unit 



Same as 1 except 
sample is pro- 
portionate to 
variability 



2. If sampling 
units are 
geograph- 
ically de- 
fined, cuts 
down field 
costs (i .e. , 
travel) 

1. Reduces var- 
iability 



1. Assures re- 
presenta- 
tiveness with 
respect to 
property 
which forms 
basis of 
classifying 
units; there- 
fore yields 
less varia- 
bility than 

A or C 

2. Decreases 
chance of 
failing to 
include mem- 
bers of pop- 
ulation be- 
cause of 
classifica- 
tion process 

3. Characteris- 
tics of each 
stratum can 
be estimated, 
and hence 
comparisons 
can be made 

1. Less varia- 
bility for 
same sample 
size than 1 



1. Lack of know- 
ledge of size 
of each sam- 
pling unit be- 
fore selection 
increases var- 
iability 



1. Requires ac- 
curate infor- 
mation on pro- 
portion of 
population in 
each stratum, 
otherwise 
increases er- 
ror 

2. If stratified 
lists are not 
available, may 
be costly to 
prepare them; 
possibility of 
faulty classi- 
fication and 
hence increase 
in variability 



1. Requires know- 
ledge of varia 
bility of per- 
tinent charac- 
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APPENDIX 1 (continued) 



3. Dispro- 
portion- 
ate 



E. Cluster 



within strata 
as well as their 
size 

Same as 1 except 
that size of 
sample is not 
proportionate 
to size of sam- 
pling unit but 
is dictated by 
analytical con- 
siderations or 
convenience 



1. More effi- 
cient than 1 
for compari- 
son of stra- 
ta or where 
different 
errors are 
optimum for 
different 
strata 



teristic with- 
in strata 

1. Less efficient 
than 1 for de- 
termining pop- 
ulation char- 
acteristics ; 
i . e . , more 
variability for 
same sample 
size 



1. Larger errors 
for comparable 
size than other 
probability 
samples 

2 . Requires abil- 
ity to assign 
each member of 
population 
uniquely to a 
cluster; in- 
ability to do 
so may result 
in duplication 
or omission of 
individuals 



Select sampling 
units by som.e 
form of random 
sampling; ul- 
timate units 
are groups; 
select these at 
random and take 
a complete count 
of each 



1. If clusters 
are geo- 
graphically 
defined, 
yields low- 
est field 
costs 

2 . Requires 
listing only 
individuals 
in selected 
clusters 

3 . Character- 
istics of 
clusters as 
well as 
those of pop- 
ulation can 
be estimated 

4. Can be used 
for subse- 
quent sam- 
ples, since 
clusters , 
not indivi- 
duals , are 
selected, 
and substi- 
tution of in- 
dividuals 
may be per- 
missible 



APPENDIX 1 (continued) 



F. Strati- 
fied 
Cluster 


Select clusters 
at random from 
every sampling 
unit 


1 . Reduces var- 
iability of 
plain cluster 
sampling 


1. Disadvantages 
of stratified 
sampling added 
to those of 
cluster sam- 
pling 

2. Since cluster 
properties may 
change , advan- 
tage of strati- 
fication may 

be reduced and 
make sample 
unusable for 
later research 


G. Repeti- 
tive : 
multiple 
or se- 
quential 


Two or more sam- 
ples of any of 
the above types 
are taken, using 
results from 
earlier samples 
to design later 
ones, or deter- 
mine if they are 
necessary 


1. Provides es- 
timates of 
population 
character- 
istics which 
facilitate 
efficient 
planning of 
succeeding 
sample, 
therefore 
reduces er- 
ror of final 
estimate 

2. In the long 
run reduces 
number of 
observations 
required 


1. Complicates 
administration 
of field work 

2 . More computa- 
tion and anal- 
ysis required 
than in non- 
repetitive sam- 
pling 

3. Sequential sam- 
pling can only 
be used where 

a very small 
sample can ap- 
proximate re- 
presentative- 
ness and where 
the number of 
observations 
can be in- 
creased con- 
veniently at 
any stage of 
the research 


H . Judgment 


Select a subgroup 
of the popula- 
tion which, on 
the basis of 
available infor- 
mation can be 
judged to be 
representative 


1. Reduces cost 
of preparing 
sample and 
field work, 
since ulti- 
mate units 
can be se- 
lected so 


1. Variability 
and bias of es- 
timates cannot 
be measured or 
controlled 

2. Requires strong 
assumptions or 
considerable 
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APPENDIX 1 (continued) 





of the total 
population; take 
a complete count 
or subsample of 
this group 


that they 
are close to- 
gether 


knowledge of 
population and 
subgroup se- 
lected 


1. Quota 


Classify popula- 
tion by perti- 


1. Same as 
above 


1. Introduces bias 
of observers' 




nent properties ; 
determine de- 
sired proportion 
of sample from 
each class; fix 
quotas for each 
observer 


2 . Introduced 
some strati- 
fication ef- 
fect 


classification 
of subject and 
non- random 
selection with- 
in classes 
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APPENDIX 2 



SIMPLIFIED TAXONOMY OF EXPERIMENTAL STATISTICS 



Type of 
Data 




Type of 


Samples 


















One -sample 


Two-Samples 


k-Samples 






Matched 


Random 


Matched 


Random 


Metric 1 - 




Matched t- 


Random t- 


Two-way 


One-way 






test 


test 


Anal, of 


Anal, of 


T 




(D 133) 


(D 140) 


variance 


variance 


R 




(E2 278) 


(El 90) 


(El 158) 


(D 178) 


E 




(SE 485) 


(M 102) 


(M 303) 


(El 117) 


A 




(M 101) 


One-way 




(M 265) 


T 




Two-way 


Anal, of 




Anal, of 


M 




Anal, of 


variance 




Covariance 


E 




Variance 


(El 117) 




(El 281) 


N 




(El 158) 


(M 265) 




(M 362) 


T 




(M 294) 


Anal, of 
Covariance 
(El 281) 






k- 












T 








Factorial 


Factorial 


R 








designs 


designs 


E 




See K Sam- 




(El 175) 


(El 175) 


A 




ples 








T 








(M 318) 


(M 318) 


M 




and 1 




Analysis 


Latin- 


E 




Treatment 








N 








of 


square 


T 








Covariance 


(El 254) 


S 








(El 281) 


Anal, of 
Covariance 
(El 281) 


Ranked 


Kolmogorov 


Sign Test 


Median 


Friedman 


Extension 




-Smirnov 


(D 236) 


Test 


two-way 


of Median 
test 




(SI 47) 


(E2 288) 


(D 239) 


anal . of 


(E2 389) 




one -sample 


(M 376) 


(E2 387) 


variance 


(M 376) 




runs test 


(SE 441) 


(M 376) 


(E2 402) 


(SE 446) 




(M 238) 


(SI 68) 


(SE 435) 


(M 378) 


(SI 179) 




(SE 432) 


Wilcoxin 


(SI 111) 


(SE 450) 


Kruskal- 




(SI 52) 


Matched 


Mann-Whit- 


(SI 166) 


Wallis One- 






Pairs 


ney U 




Way Anal- 






Ranks 






ysis 






Test 


(D 240) 




of Variance 
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(D 237) 


(E2 417) 




(D 247) 




(E2 291) 


(M 377) 




(E2 423) 




(SE 489) 


(SE 437) 




(M 378) 




(SI 75) 


(SI 116) 




(SE 450) 






Wald-Wolf- 
owitz Runs 




(SI 184) 






(D 244) 
(SI 136) 







SIMPLIFIED TAXONOMY OF MEASURES OF RELATIONSHIP 

Y-VARIABLE SCALE 



X-SCALE 


METRIC 


ORDINAL 


NOMINAL 


METRIC 

1- variable 


biserial r 
Pearson r 
tetrachoric r 
triserial r 


Index of order 
assoc. 

Kendall's Tau 
Spearman rho 


Eta 

Point biserial 


k-variables 


Eta 

Multiple bi- 
serial 
Multiple r 
partial r 


coef. of con- 
cordance 
Eta 


Eta 

contingency coef. 


ORDINAL 

1- variable 


Index of order 
assoc. 

Eta 

Kendall's tau 
Spearman rho 


Kendall's tau 
Index of order 
assoc. 

Spearman rho 


phi (dichotomous) 


k-variables 


coef. of con- 
cordance 


coef. of con- 
cordance 


contingency coef. 


NOMINAL 

1-variable 


point biserial 


phi (dicho- 
tomous ) 


phi 


k-variables 


contingency 

coef. 


contingency 

coef. 


contingency coef. 



er|c 
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MEASURE REFERENCE 





Downie 


Green 

Edwards 


McNemar 


Senders 


Siegel 


biserial r 


193 


188 


189 




^ 


coef. of concordance . . . 


209 


402 


379 




229 


contingency coef 


210 


381 


198 


— — — 


196 


Eta 


199 


197 


202 


229 


... 


Index of order assoc. . . 


— 


— 


— 


130 


... 


Kendall's tau 


208 


— 


— 




213 


multiple r 


205 


— 


169 


— — — 


— 


multiple biserial r ... 


— 


— 


205 


— 


— 


partial r 


203 


tr» 


164 


— 


— 


Pearson r 


78 


142 


112 


242 


... 


phi 


196 


185 


197 


— 


... 


point biserial r 


189 


182 


192 


— — — 


... 


Spearman rho 


206 


193 


203 


133 


202 


tetrachoric r 


198 


190 


193 


— 


... 


triserial 


194 


" 


“ “ 




... 



Type of 
Data 




Type of 


Samples 








One -sample 


Two-Samples 


k-Samples 






Matched 


Random 


Matched 


Random 




Binomial Test 
(El 23, 48) 

(M 46) 

(SE 404) 

(SI 36) 

Chi^ one- 
sample test 
(El 64) 

(E2 221) 

(SE 402) 

(SI 42) 


McNemar 
Test for 
Changes 
(El 57) 

(M 52, 224) 
(SI 63) 


t-test 
for pro- 
portions 
(El 51) 

(M 50) 
Exact ' 
proba- 
bility 
test 
(El 55) 
(SI 96) 

Chi^ two- 
sample 
test 
(El 69) 
(E2 367) 
(SE 418) 
(SI 104) 


Cochran 
Q test 
(SI 161) 


Chi^ for 
k inde- 
pendent 
samples 
(El 65) 
(E2 372) 
(SE 402) 
(SI 175) 
(M 230) 
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El Edwards, Exp, Design in Psychological Research 

E2 Edwards, Statistical Methods for the Behavioral Sciences 

D Downie and Heath, Basic Statistical Methods 

M McNemar, Quinn, Psychological Statistics 

SE Senders, V. L. , Measurement and Statistics^ A Basic Text 

Emphasizing Behavioral Science Applications 

SI Siegel, Sidney: Eonparametric Statistics for the Behav- 

ioral Science 
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